History learning with textual or visual tasks: student dialogue

Multimodal representations are representations containing a combination of text and schemas and/or pictures. According to the Cognitive Theory of Multimedia Learning such representations can be powerful learning tools. The study described here approaches this theory from the domain of history in co-construction tasks. In an experimental study, the dialogues of pupils who co-constructed either textual representations or multimodal representations integrated in a timeline were compared. The participants were 12 to 14-yearold pupils in pre-vocational secondary education who worked in dyads on a series of four history tasks. Dialogue protocols of the taped student conversations for one of these tasks were analysed. The results show that integrated multimodal representations do – to some extent – lead to more discussion about domain content as well as about procedural issues than working with textual representations. Theoretical framework Multimodal representations combine two or more modes of representation, for example, a verbal text with one or more types of visualisation (i.e., schematic and/or depictive). While there has been a fair amount of research on the effects of learning with multimodal representations – in particular visualisations presented with texts – there has been less research on the learning processes that occur when using multimodal representations in tasks. This study sheds light on differences in the learning process between multimodal and verbal tasks at the level of task content. There is a large body of research on learning with multimodal representations (Ainsworth, 1999; Larkin & Simon, 1987; Mayer & Chandler, 2001; Schnotz, 1993). In these studies multimodal representations are seen as cognitive thinking tools – tools for remembering, thinking, and problem solving (Jonassen, Reeves, Hong, Harvey, & Peters, 1997) – and the focus is primarily on the learning outcomes that these representations produce. These outcomes are explained by Dual Coding Theory (Paivio, 1991), Cognitive Theory of Multimedia Learning (Mayer, 2001) and/or Cognitive Load Theory (Kirschner, 2002; Sweller, Van Merriënboer & Paas, 1998). In recent research, two topics related to learning with multimodal representations have gained attention, namely: 1) the effects of constructing different forms of multimodal representations on learning processes and outcomes, and 2) the learning and interaction processes that are provoked and supported by the construction of multimodal representations by the learners themselves. We will first elaborate on these topics below. Cox (1999) argues that constructing external representations can be beneficial for learners. Several studies have shown positive learning effects for one specific type of task in which a multimodal representation is constructed, namely a concept map (e.g., Horton, McConney, Gallo, Woods, Senn, & Hamelin, 1993; O’Donnell, Dansereau, & Hall, 2002; Van Boxtel, Van der Linden, & Kanselaar, 2000). Concept maps are diagrams that indicate interrelationships between concepts and represent conceptual frameworks within a specific domain of knowledge (Novak, 1990). In order to construct a concept map, students have to think about and describe both concepts that are instrumental to a specific situation and the relationships between those concepts. By doing this, concept mapping helps students focus on the macrostructure of the content, stimulates elaboration and can provide multiple retrieval paths for accessing knowledge. The higher level of activity required from students in construction of representations leads to different learning processes. According to Bodemer, Ploetzner, Bruchmüller, and Häcker (2005), students often remain rather passive when they only have to look at a multimodal representation. The researchers argue that active integration of textual and pictorial information by the students themselves is essential for successful learning. They conducted an experimental study in which the students had to relate textual and pictorial information about the working of a tire pump by actively dragging and dropping captions into a drawing of a tire pump on a computer screen. This active integration significantly improved learning. Van Meter and Garner (2005) give examples of tasks in which students construct visual representations, such as organizing or sequencing, given component pieces of a representation. These ideas about the level of activity and active integration of representations led to an experimental study that was conducted to assess the value of active construction of multimodal representations of historical phenomena (Prangsma, Van Boxtel, & Kanselaar, 2008). Students constructed or completed multimodal representations for different historical phenomena and developments in the Early Middle Ages along with a coordinating timeline. Then, the learning outcomes of pupils who co-constructed textual representations were compared with learning outcomes of pupils who co-constructed multimodal representations integrated in a timeline. Results showed that working on multimodal representations integrated in a timeline led to higher learning outcomes than co-constructing textual representations. The differences in the content of the discourse of collaborating students in these two conditions were assumed to have contributed to this outcome. Studies on the construction of multimodal representations in collaborative settings that include discourse analysis give some insights into the learning processes with multimodal tasks (Bodemer, Ploetzner, Bruchmüller, & Häcker, 2005; Suthers & Hundhausen, 2003; Van Boxtel, Van der Linden, Roelofs, & Erkens, 2002). These studies showed that multimodal representations can facilitate discourse – or articulation – at the content level, thus encouraging the use of domain-specific language, and ultimately knowledge building. Discourse analysis in these studies has focused on the communicative functions (Suthers & Hundhausen), on the activities in the software (Bodemer et al.), on procedures and task management (Erkens, Jaspers, Prangsma, & Kanselaar, 2005), and on topic content (Fischer, Bruhn, Gräsel, & Mandl, 2002; Roth & Roychoudhury, 1994; Van Boxtel, Van der Linden, & Kanselaar, 2000; Van Drie, Van Boxtel, Jaspers, & Kanselaar, 2005).